Topic Detection, Tracking, and Trend Analysis Using Self-Organizing Neural Networks

We address the problem of Topic Detection and Tracking (TDT) and subsequently detecting trends from a stream of text documents. Formulating TDT as a clustering problem in a class of self-organizing neural networks, called the Adaptive Resonance Theory (ART) networks, we propose an incremental algorithm to solve this clustering problem. From the topics being detected and tracked, we show how trends can be identi ed. From our experimental studies, we nd that our algorithm has been able to detect hot topics automatically and track them to a good accuracy. The method is also observed to enable discovering interesting trends that are not directly mentioned in the individual documents but deducible only from reading all relevant documents.